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JOB  ASSESSMENT  SOFTWARE  SYSTEM  (JASS)  FOR  ANALYSIS  OF  WEAPON  SYSTEMS 
PERSONNEL  REQUIREMENTS:  RESEARCH  REPORT 


EXECUTIVE  SUMMARY 


Requirements : 

In  order  to  integrate  the  impact  of  personnel  factors  into  the  design  of 
Army  weapon  systems  it  is  necessary  to  estimate  the  human  ability  or  aptitude 
requirements  of  the  system.  This  process  needs  to  be  accomplished  early  in 
the  system  design  cycle.  This  report  describes  the  initial  development  of  a 
technique  to  meet  these  goals . 


Procedure: 

Fleishman's  taxonomy  of  40  cognitive,  perceptual,  and  psychomotor  abili¬ 
ties  was  programmed  onto  a  microcomputer.  This  computerized  version  was  de¬ 
signed  to  allow  users  to  respond  to  the  taxonomy  quickly  and  efficiently. 

The  resulting  output  is  a  profile  of  the  ability  requirements  for  the  par¬ 
ticular  job  that  was  being  analyzed. 


Findings : 

The  ability  taxonomy  was  successfully  implemented  on  the  microcomputer 
and  this  implementation  is  described  in  the  report.  Further  research  is 
needed  to  test  and  refine  the  computerized  procedure  before  it  can  be  used 
on  an  operational  basis. 
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WEAPON  SYSTEMS  PERSONNEL  REQUIREMENTS:  RESEARCH  REPORT 

Background 

The  Army  currently  has  underway  a  large-scale  modernization  program 
which  is  expected  to  result  in  the  fielding  of  over  600  new  weapon  systems 
during  the  next  decade.  Historically,  the  decision  to  field  or  not  to  field  a  new 
weapon  system  has  been  based  on  the  criteria  of  hardware  cost  and  performance. 
Often  overlooked  was  the  "human  resource  cost"  of  new  systems,  tb  is,  the 
number  of  personnel  required  to  operate  and  maintain  them,  and  th  ptitudes 
and  skills  that  those  personnel  must  possess.  Over  the  past  sevetai  years, 
however,  the  Army  has  become  acutely  aware  that  the  human  resource  cost  of 
new  weapon  systems  can  no  longer  be  ignored  (e.g.,  Kerwin  &  Blanchard,  1980). 

Three  factors,  operating  concurrently,  are  responsible  for  this  increased 
awareness.  First,  census  data  indicate  that  the  quantity  of  individuals  available 
for  military  service  (18-25  year  olds)  will  decline  throughout  this  century  and,  if 
the  birth  rate  remains  unchanged,  for  the  foreseeable  future.  Second,  the 
aptitudes  and  skills  some  new  recruits  bring  into  the  Army  appear  to  be  on  the 
decline.  Standardized  aptitude  and  achievement  test  scores  have  shown  a 
consistent  decline  over  the  past  15  years  (Walters,  Eitelberg,  If  Laurence,  1981). 
Taken  together,  these  two  factors  indicate  increased  future  competition  among 
the  Armed  Forces  and  the  civilian  sector  for  qualified  personnel.  The  competi¬ 
tion  is  expected  to  be  the  most  severe  for  the  more  highly  skilled  individuals. 

The  third  factor  is  the  increasing  technological  sophistication  of  the 
Army's  new  weapon  systems.  It  is  widely  accepted  that  increased  sophistication 


is  increasing  operator  and  maintainer  job  complexity  and,  in  turn,  increasing  skill 
requirements  and  quantitative  demand  for  personnel  (Kerwin  &  Blanchard,  1980). 
The  Army,  therefore,  faces  the  possibility  of  increasing  quantitative  and 
qualitative  personnel  demands  while  the  capability  of  the  population  to  fill  these 
demands  is  decreasing.  It  follows  that  the  human  resource  cost  must  become  a 
factor  in  acquisition  decisions.  This,  in  turn,  requires  the  development  of 
techniques  which  can  provide  accurate  estimates  of  manpower  and  personnel 
requirements  early  in  the  acquisition  process.  Such  techniques  will  permit 
comparison  of  human  resource  demand  to  supply  and  allow  the  use  of  human 
resource  data  as  criteria  in  weapon  system  design.  Such  data  could  provide  the 
means  for  making  judgments  about  the  impact  of  design  alternatives  on  the 
human  resource  pool,  as  well  as  appraisals  concerning  what  constraints  the 
human  resource  pool  may  impose  on  weapon  system  design  alternatives  (Askren, 
1976). 

Problem 

Weapon  system  design  begins  with  a  statement  of  the  purpose  of  the 
system,  that  is,  one  or  more  "missions"  the  system  is  expected  to  perform. 
Following  the  determination  of  the  mission  requirements,  a  conceptual  design 
can  be  formulated  for  the  new  weapon  system.  Inherent  in  such  a  design  concept 
is  an  allocation  of  functions  between  human  and  machine.  Given  the  specifica¬ 
tion  of  the  functions  to  be  performed  by  the  human  in  the  system,  it  is  possible 
to  break  down  those  functions  into  component  tasks.  The  problem  at  hand  is  to 
develop  valid  techniques  that  can  be  used  by  the  system  design  team  to  translate 
these  tasks  into  meaningful  behavioral  components,  and  then  to  obtain  a 
quantitative  estimate  of  the  required  amount  of  each  component.  This  informa¬ 
tion  can  then  be  used  for  comparisons  to  the  human  resource  supply  ,  pool  as 
outlined  above.  This  process  is  depicted  schematically  in  Figure  1. 
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FIGURE  ^titude  Assessment  Within  the  Context  of  System  Design 


The  human  aptitude  requirement  of  a  system  is  a  key  component  necessary 
in  making  supply-side  comparisons.  Since  the  term  "aptitude"  has  several 
meanings  (e.g.,  Dunnette,  1976,  and  Fiejshman,  1975),  it  is  important  to  define 
its  use  in  this  context.  Aptitude  will  be  used  to  refer  to  a  general  characteristic 
of  an  individual  that  affects  task  performance.  Aptitudes  are  assumed  to  be 
enduring  traits  that  are  difficult  or  impossible  to  alter  through  cost-effective 
training.  Consequently,  aptitude  requirements  should  be  of  crucial  consideration 
during  weapon  system  development  since  any  discrepancy  between  the  aptitudes 
required  to  operate  and  maintain  a  system  and  those  present  in  the  available 
personnel  will  be  very  difficult  to  overcome. 

This  report  describes  the  initial  development  of  a  structured  and  time- 
efficient  rating  scheme  that  has  potential  use  for  weapon  system  designers  to 
estimate  the  human  aptitude  requirements  of  weapon  systems  during  the 
conceptual  stage  of  development. 

Basic  Approach 

The  foundation  of  this  rating  technique  is  the  extensive  research  by 
Fleishman  (1972,  1975)  who  identified  basic  human  aptitudes  or  abilities  and 
their  relationship  to  performance  on  a  wide  range  of  tasks.  This  research  has 
resulted  in  a  taxonomy  of  forty  cognitive,  perceptual,  and  psychomotor 
aptitudes.  The  perceptual  and  psychomotor  aptitudes  in  this  taxonomy  stem 
from  Fleishman's  own  factor  analytic  research,  while  the  cognitive  aptitudes  are 
taken  largely  from  the  Educational  Testing  Service's  manual  for  factor- 
referenced  cognitive  tests  (Ekstrom,  French,  Harman,  6r  Dermen,  1976). 

The  Fleishman  approach  was  selected  over  other  behavioral  taxonomies 
(i.e.,  Siegel,  1980)  because  his  basic  procedure  has  been  extended  (Mallamad,  et 
al.,  1980)  into  a  binary  decision-flow  structure  which  reduces  the  information 
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processing  and  decision-making  demands  on  the  analyst.  This  structure  contains 
questions  and  task  examples  that  suggest  the  presence  or  absence  of  an  aptitude, 
and  help  differentiate  a  given  aptitude  from  similar  aptitudes.  The  key 
advantage  of  this  structure  is  that  it  lends  itself  easily  to  computer  imple¬ 
mentation.  A  computerized  technique  was  desirable  to  provide  flexibility,  ease 
of  use,  and  rapid  analysis  of  group  and  individual  data. 

Besides  the  advantages  noted  above,  a  computerized  approach  is  able  to 
overcome  one  of  the  major  problems  in  using  the  paper  and  pencil  version  of  the 
flow  diagrams.  Mallamad,  et  al.  (1980)  noted  that  the  flow  diagrams  were  useful 
only  in  identifying  the  need  for  a  particular  aptitude,  and  that  some  other 
technique  is  required  to  determine  the  level  of  the  aptitude.  This  determination 
can  be  easily  accomplished  via  computer  by  having  "yes"  responses  to  questions 
in  the  flow  structure  lead  to  an  aptitude  rating  scale  like  the  one  shown  in  Figure 
2.  This  scale  (with  quantified  anchor  points,  previously  established)  is  used  to 
quantitatively  rate  the  relative  level  of  a  particular  aptitude  required  to  perform 
the  job  or  task  being  analyzed. 

This  report  describes  a  preliminary  computerized  job  assessment  system 
which  uses  a  standard  portable  CRT  display  and  off-the-shelf  microcomputer 
components.  This  system  is  named  the  Job  Assessment  Software  System  (JASS). 
The  software  is  modular  in  format  and  consists  of  three  basic  elements:  (1)  The 
binary  decision  flow  branching  network  for  aptitude  specification;  (2)  The 
capability  to  produce  and  display  a  variety  of  rating  scales  and  task  examples  for 
aptitude  rating;  (3)  Data  aggregation,  processing,  reduction,  and  analysis  routines 
to  summarize  system  aptitude  requirements. 

The  System  Requirements  Section  describes  the  development  of  the 
software  logic  system  and  the  preliminary  rating  scales;  the  Implementation 
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FIGURE  2:  Example  of  Aptitude  Rating  Scale 
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Procedures  Section  summarizes  the  software  and  system  programs;  the  Initial 


User  Acceptability  Section  discusses  the  initial  industry  reaction  to  the  system; 


and  the  final  section,  Future  Directions,  discusses  further  requirements  for 


research  and  system  refinement. 


SYSTEM  REQUIREMENTS 


The  development  of  the  computerized  system  required  two  psychological 


foundations:  ‘a  workable  taxonomy  of  human  aptitudes,  and  a  set  of  behavioral 


anchor  statements  to  help  quantify  the  level  of  an  aptitude  that  is  required. 


Both  of  these  requirements  are  discussed  in  turn  below. 


Aptitude  Taxonomy 


Fleishman  (1972,  1975)  identified  forty  basic  human  abilities  or  aptitudes 


that  underlie  performance  on  job  tasks.  These  aptitudes  are  listed  in  Table  1. 


Table  1:  Fleishman's  Aptitude  Taxonomy 


•  Oral  Comprehension 

•  Written  Comprehension 

•  Oral  Expression 

•  Written  Expression 

•  Memorization 

•  Problem  Sensitivity 

•  Originality 

•  Inductive  Reasoning 

•  Category  Flexibility 

•  Deductive  Reasoning 

•  Information  Ordering 

•  Math  Reasoning 

•  Number  Facility 

•  Fluency  of  Ideas 

•  Time  Sharing 

•  Flexibility  of  Closure 

•  Speed  of  Closure 

•  Selective  Attention 

•  Perceptual  Speed, 

•  Spatial  Orientation 


•  Visualization 

•  Static  Strength 

•  Explosive  Strength 

•  Dynamic  Strength 

•  Trunk  Strength 

•  Stamina 

•  Extent  Flexibility 

•  Dynamic  Flexibility 

•  Gross  Body  Equilibrium 

•  Speed  of  Limb  Movement 

•  Gross  Body  Coordination 

•  Multi-limb  Coordination 

•  Wrist-finger  Speed 

•  Finger  Dexterity 

•  Manual  Dexterity 

•  Arm/Hand  Steadiness 

•  Control  Precision 

•  Rate  Control 

•  Reaction  Time 

•  Choice  Reaction  Time 


Fleishman  contends  that  the  combination  of  these  forty  aptitudes  required 


to  perform  a  task  or  job  can  be  used  to  describe  that  task  or  job.  Therefore,  it 


should  also  be  possible  to  describe  the  personnel  requirements  for  any  weapon 
system  by  a  combination  of  the  forty  abilities. 

Mallamad,  et  al.  (1980)  extended  the  utility  of  this  approach  by  developing 
a  series  of  binary  decisions  as  to  whether  or  not  each  of  the  forty  aptitudes  is 
required  for  performing  a  job  or  task.  This  paradigm  was  adopted  and  adapted 
for  the  current  project.  The  current  3ASS  branching  network  consists  of 
transaction  blocks  and  scale  blocks.  The  transaction  blocks  elicit  a  "yes"  or  "no" 
response  from  the  rater.  A  "yes"  response  indicates  that  an  aptitude  is  required 
and  ultimately  leads  to  a  scale  block  wherein  the  rater  estimates  the  required 
magnitude  of  that  aptitude.  A  "no"  response  simply  jumps  the  rater  to  the  next 
aptitude.  An  example  of  this  process  is  shown  in  Figure  3.  In  the  example,  the 
scale  blocks  are  those  labeled  Dynamic  Strength,  Trunk  Strength,  and  Stamina. 
An  example  of  a  scale  block  was  shown  in  Figure  2. 

The  3 ASS  computer  program,  written  in  Applesoft©  BASIC  language, 
produces  the  branching  decision  network  of  which  the  sample  in  Figure  3  is  a 
part.  3ASS  performs  the  following  functions:  formats  and  displays  the  text  in 
each  of  the  flowchart  blocks,  prompts  and  processes  the  user  responses  to  the 
flowchart  blocks,  sets  up  a  user’s  file,  stores  the  aptitude  assessment  scale 
scores,  and  retrieves  this  file  if  the  user  revises  or  appends  the  job  ratings. 

The  actual  text  in  each  of  the  data  blocks  is  not  a  part  of  the  computer 
program.  The  text  is  stored  on  the  computer  disc  as  separate  text  data  files. 
These  text  data  files  are  "executed"  into  the  program  as  directed  by  the  user's 
response.  In  other  words,  the  main  program  actually  changes  as  required  to 
incorporate  the  information  in  the  text  data  file.  For  example,  if  the  user 
answers  "no"  to  a  transaction  block,  the  program  processes  this  information  and 
looks  for  the  text  data  file  that  follows  a  "no"  response.  Each  text  data  file 
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contains  not  only  the  flow  block  text  but  the  numbers  which  lead  to  the  next 
data  block.  The  text  data  file  contains  flags  which  indicate  what  file  is  next  if 
the  user  answers  "no"  and  what  file  is  next  if  the  user  answers  "yes."  In  this 
form,  the  program  can  be  used  with  any  set  of  text  data  files.  The  branching 
network  program  can  be  used  with  a  variety  of. test  data  files  without  changing 
the  basic  program. 

Anchor  Statement  Development 

A  drawback  of  the  Mallamad,  et  al.  (1980)  binary  decision  flowchart  was 
that  by  itself  it  does  not  produce  an  estimate  of  the  level  of  an  aptitude  required 
for  task  performance.  The  materials  developed  in  3ASS  combine  a  branching 
network  decision  flowchart  with  rating  scales  for  each  ability.  The  rating  scales 
allow  estimation  of  the  level  of  each  ability  needed  for  a  given  job.  Using  these 
scales,  a  rater  quantifies  the  required  level  of  an  aptitude  by  assigning  a  score 
between  0  and  7.  Example  task  descriptions  are  placed  on  the  scales  to  provide 
the  rater  with  comparisons  when  rating  a  specific  job.  These  task  descriptions 
are  called  "anchor  points."  In  his  work  Fleishman  (1972,  1975)  used  a  set  of 
generalized  anchor  points  (see  Figure  4)  which  are  assumed  to  be  applicable  to 
most  jobs  or  tasks.  While  these  scales  have  been  successfully  used  to  evaluate 
some  military  jobs  (Mallamad,  et  al.,  1980),  preliminary  studies  conducted  by  the 
Army  Training  and  Doctrine  Command's  (TRADOC)  Soldier  Support  Center 
suggested  that  they  are  too  general  in  nature  for  widespread  Army  use.  For  this 
reason,  new  anchor  points  were  developed  which  would  be  more  relevant  to  the 
Army  environment.  Two  different  sets  of  scales  were  proposed:  one  set 
designed  for  a  wheeled  vehicle  repairman  and  one  set  designed  specifically  for  a 
helicopter  crewman. 
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The  anchor  points  for  mechanic  scales  were  equally  spaced  on  the  scale  and 
represent  examples  of  high,  moderate,  some,  and  very  little  aptitude  demand 
levels.  This  scale  is  a  graphic  rating  scale  which  allows  the  rater  to  indicate  the 
degree  of  the  aptitude  required  by  the  task.  It  is  the  simplest  and  most  common 
type  of  rating  scale.  An  example  mechanic's  scale  is  shown  in  Figure  5. 

The  job-relevant  anchor  points  were  added  to  the  mechanic  scales  through 
a  structured,  although  subjective,  procedure.  The  job  of  mechanic  was  selected 
as  an  example  for  initial  program  development  because  the  developers  were 
somewhat  familiar  with  the  job  requirements.  A  determination  was  then  made 
of  the  variable  factors  in  the  activity  which  defined  the  degree  of  difficulty. 
The  authors  of  the  anchor  points  first  tried  to  identify  an  activity  associated 
with  each  ability  which  could  be  put  on  a  continuous  scale  of  difficulty.  The 
range  of  activity  difficulty  was  divided  into  three  equal  parts,  using  example 
tasks  representing  four  difficulty  levels:  high  amount,  moderate  amount,  some, 
very  little.  Example  tasks  were  drawn  from  the  authors'  experience. 

However,  the  authors  were  not  sufficiently  experienced  mechanics  to  be 
able  to  develop  meaningful  job  examples  for  all  aptitudes.  This  points  out  the 
problem  of  this  approach  to  anchor  development.  As  pointed  out  earlier, 
preliminary  studies  indicated  the  desirability  of  job-relevant  anchor  points.  The 
mechanic's  scale  anchor  points  were  developed  on  the  basis  of  factors  inherent  in 
the  aptitude  itself.  Job  relevant  examples  were  added  when  possible.  If  one, 
two,  or  three  authors  had  prepared  the  anchors  purely  from  the  basis  of  job 
related  tasks,  personal  biases  would  have  undoubtedly  affected  the  results. 
Clearly,  job  specific  anchor  points  should  be  prepared  by  a  statistically  signif¬ 
icant  consensus  of  subject  matter  experts.  Because  of  these  shortcomings,  the 
current  mechanics  scales  are  usable  only  as  place  holders  during  the  software 
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development  and  for  some  preliminary  testing.  New  scales  are  needed  for 
operational  use. 

In  his  work,  Fleishman  uses  a  set  of  generalized  anchor  statements  which 
are  assumed  to  be  relevant  across  all  or  most  jobs  and  tasks.  Such  generalized 
anchors  have  two  major  advantages.  First,  they  are  based  upon  considerable 
research  with  large  sample  sizes,  so  they  should  be  accurate  and  stable;  second, 
use  of  generalized  scales  should  allow  simple  and  straightforward  comparisons  of 
aptitude  requirements  across  Army  systems.  The  generalized  scales  may  present 
a  serious  disadvantage,  however.  In  an  unpublished  work,  the  Soldier  Support 
Center  (SSC)  used  generalized  aptitude  ratings  in  a  survey  format  as  part  of  the 
Soldier  Machine  Interface  Requirements  (SMIR)  study  advisory  group's  investiga¬ 
tion.  This  research  was  terminated  before  completion  and  subjected  to  consider¬ 
able  criticism.  The  basis  of  this  criticism  was  the  belief  that  the  general  anchor 
statements  were  not  relevant  to  Army  jobs  and  would  be  difficult  for  Army 
personnel  to  understand.  While  the  basic  issue  of  general  versus  specific  anchors 
can  only  be  resolved  with  further  research,  it  was  decided  that  the  initial  testing 
of  the  JASS  procedure  should  use  anchor  statements  that  are  directly  relevant  to 
an  Army  job.  It  was  believed  that  this  decision  would  increase  the  Army's 
acceptance  of  the  procedure.  The  job  chosen  for  analysis  was  that  of  a 
helicopter  crewman. 

The  anchor  points  for  the  helicopter  crewman's  scales  were  adapted  from 
the  aptitude  scales  used  by  Rossmeissl  and  Dohme  (1982).  The  scales  in  this 
study  were  based  upon  a  modified  list  of  Fleishman's  (1975)  aptitudes  (see 
Appendix  B  for  definitions  of  the  aptitudes  that  were  used).  The  scaling 
technique  utilized  was  similar  to  that  which  is  used  to  jvelop  behaviorally 


anchored  descriptions  of  job  behavior  (Smith  &  Kendall,  1963)  or  weighted 
behavioral  checklists  (Knauft,  1948). 

Four  Army  helicopter  missions  (aeroscout,  attack,  cargo,  and  utility)  were 
studied  to  determine  the  abilities  that  underlie  successful  mission  performance. 
Thirty  aptitudes  were  identified  as  possible  requirements  in  the  task  analysis  of 
the  four  missions.  To  develop  aviation-specific  anchors  for  these  30  aptitudes, 
an  ARI  psychologist  and  an  ARI  Master  Aviator  developed  as  many  Army 
aviation  task  statements  as  possible  for  each  aptitude,  the  objective  being  to 
create  anchor  statements  that  would  cover  as  much  as  possible  the  range  of  each 
aptitude  from  the  least  to  the  greatest  level  required  for  performing  all  four 
Army  aviation  missions.  In  other  words,  mission  general  statements  were 
developed  that  were  common  to  all  four  missions.  For  each  aptitude,  fifteen  to 
twenty  candidate  anchor  statements  were  generated  using  the  Aircrew  Training 
Manuals  ar.d  Helicopter  Operator's  Manuals  (Dash  Tens)  as  guides. 

Once  the  candidate  anchor  statements  were  generated,  two  Standardiza¬ 
tion  Instructor  Pilots  (SIPs)  were  brought  in  to  represent  each  mission  and  a 
roundtable  discussion  was  held  to  eliminate  those  statements  that  were  judged 
not  to  apply  to  all  four  missions.  Certain  mission  oriented  statements  were  also 
eliminated  because  they  are  not  part  of  the  training  regimen  for  a  given  mission. 
In  addition,  the  eight  SIPs  edited  the  wording  of  the  statements  to  improve  their 
clarity.  This  process  reduced  the  number  of  anchor  candidates  down  to  288. 

The  remaining  anchor  statements  were  included  in  an  evaluation  package 
that  was  administered  to  44  field  experienced  Army  Warrant  Officer  aviators. 
These  subjects  comprised  31  current  field  aviators  from  Fort  Hood  and  13 
students  in  the  Warrant  Officer  Senior  Course  at  Fort  Rucker.  The  subjects,  who 


were  mostly  CW3  and  CW4  ranks,  were  distributed  across  the  four  missions  as 
follows:  aeroscout  20%,  attack  27%,  utility  23%,  and  cargo  30%. 


The  specific  rating  procedure  was  adapted  from  the  methodology  developed 
by  Fleishman  (1975).  The  subjects  assigned  a  value  from  1-7  to  each  statement 
corresponding  to  the  amount  of  the  given  aptitude  required  to  perform  the 
aviation  mission.  Conceptual  definitions  were  provided  for  each  aptitude.  The 
mean  and  standard  deviation  for  each  of  the  288  candidate  statements  were 
calculated  and  an  attempt  was  made  to  select  three  anchors  for  each  aptitude: 
one  high,  one  low,  and  one  medium.  In  a  few  cases  (6  of  the  30  abilities)  it  was 
not  possible  to  develop  three  anchors  because  the  mean  values  clustered  toward 
one  end  of  the  seven  point  scale,  so  only  two  anchors  were  created.  For  each 
aptitude,  the  criterion  was  to  select  anchors  that  had  small  standard  deviations, 
preferably  1.5  or  less.  The  anchors  were  selected  to  obtain  the  highest  and 
lowest  mean  ratings  having  small  standard  deviations  and  also  the  rating  closest 
to  midscale  (3.5)  having  a  small  standard  deviation.  An  example  of  these  scales 
is  shown  in  Figure  6.  Rossmeissl  and  Dohme  (1982)  found  that  these  scales 
successfully  discriminate  among  the  aptitudes  required  to  operate  the  new 
aeroscout  helicopter  and  showed  reasonable  inter-rater  reliability. 

The  JASS  system  can  be  used  in  this  procedure  to  develop  meaningful,  job- 
specific  anchor  points.  The  computer  is  used  to  implement  the  methodology  used 
by  Rossmeissl  and  Dohme  (1982),  with  a  combination  of  traditional  and  modified 
Fleishman  (1975)  procedures  to  obtain  anchor  points.  Six  to  ten  job  task 
examples  are  prepared  for  each  aptitude.  Subjects  matter  experts  rate  each  task 
example  for  the  amount  of  aptitude  required.  The  ratings  are  made  in  this  case 
using  the  computer  video  screen  and  keyboard.  Data  can  then  be  saved  and 
analyzed  by  the  computer.  Tasks  are  selected  as  anchor  points  based  on  the 
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consensus  of  the  raters,  with  low  variability  tasks  being  the  anchor  points  of 
choice.  At  the  present  time,  two  sets  of  example  tasks  have  been  developed  and 
entered  into  JASS:  mechanic  and  mechanized  infantry  vehicle  crewman.  These 
tasks  can  now  be  rated  by  subject  matter  experts  knowledgeable  in  each  of  these 
two  jobs.  The  results  will  provide  objective  ‘anchor  points  for  the  jobs  of 
mechanic  and  mechanized  infantry  vehicle  crewman. 

IMPLEMENTATION  PROCEDURES 
Software  System  Overview 

This  chapter  briefly  summarizes  the  current  systems  capabilities  and 
operational  procedures. 

JASS  is  designed  for  use  by  weapon  systems  developers,  i.e.,  human  factors 
analysts,  design  engineers,  the  integrated  logistics  support  group,  and  military 
system  planners.  The  software  contains  programs  which  develop  scaling  anchors, 
collect  job  assessment  data  using  the  scaling  anchors,  and  provide  summary 
statistics  from  several  people  assessing  a  job. 

Use  of  JASS  requires  the  following  equipment:  an  Apple  ®  II  Computer, 
monitor,  two  disc  drive  units,  and  the  JASS  discs. 

The  following  is  a  brief  description  of  the  programs  that  make  up  JASS.  A 
full  description  is  contained  in  the  user's  manual  (Attachment  A). 

1.  Job  Assessment  - 

This  program  allows  the  user  to  select  and  rate  the  aptitudes 
required  to  perform  the  job.  The  program  follows  a  branching 


flowchart  as  shown  in  Figure  3.  The  user  can  either  use  the 
game  paddles  or  keyboard  to  interface  with  the  computer.  The 
user  can  quit  at  any  point  and  come  back  later  and  pick  up 
where  he  left  off.  Several  users  can  save  their  file  with'  a 


common  job  name  and  average  job  rating  scores  can  be  tallied 
and  displayed. 

Dob  Assessment  Review,  By  Rater  - 

This  program  allows  an  individual  to  review  his  job  rating  file. 
The  aptitudes  selected  are  displayed  in  order  of  the  degree 
required  to  perform  the  job.  The  aptitudes  which  are  not 
selected  as  being  requirements  for  the  job  are  also  displayed. 
Dob  Assessment  Review,  Tally  - 

This  program  presents  the  summary  statistics  of  a  group  (up  to 
UO)  of  people  rating  the  same  job.  Aptitudes  are  ranked  and 
displayed  according  to  their  average  scores.  Mean  score, 
number  of  subjects  rating  the  aptitude,  and  standard  deviations 
can  also  be  displayed  on  request. 

Dob  Assessment  Revision  - 

This  program  allows  the  individual  rater  to  change  or  update 
any  aptitude  rating  without  having  to  repeat  the  entire  job 
assessment  program. 

Anchor  Point  Development  - 

This  program  provides  a  research  tool  for  developing  meaning¬ 
ful  anchor  points  for  scaling  aptitudes.  A  subject  matter  expert 
is  first  presented  with  an  aptitude  definition  and  is  then  asked 
to  score  a  series  (approximately  8)  of  job  tasks  for  the  degree 
of  the  aptitude  required.  These  job  tasks  are  generated  by 
individuals  familiar  with  the  job.  They  are  presented  as 
examples  of  tasks  which  require  the  aptitude.  The  computer 
accurrr’lates  the  scores  of  each  subject.  The  rater  can  "skip" 


the  task  if  he  or  she  does  not  understand  it  or  feels  that  the 
aptitude  is  not  required  to  perform  it.  "Skips"  are  also 
recorded. 

6.  Anchor  Point  Development  Scores  - 

This  program  is  used  with  the  anchor  point  development  pro¬ 
gram  and  allows  the  user  to  review  the  rating  statistics  for  any 
one  of  the  aptitude  ancho  -  point  tasks.  Statistics  presented 
are:  number  of  subjects  rating  the  task,  sum  of  the  rating 
scores,  sum  of  the  rating  scores  squared,  sum  of  the  squared 
rating  scores,  and  the  number  of  "skip"  responses. 

A  user's  guide  has  been  developed  for  the  administrator  of  3ASS  (see 
Appendix  A).  The  guide  describes  3ASS  characteristics  and  requirements,  the 
Apple  reference  documentation,  disc  care  and  handling,  the  software  system,  the 
file  structure,  and  the  current  career  field  packages.  It  also  includes  a  chart 
showing  the  3ASS  programs  and  their  interrelationships,  a  reference  sheet 
summarizing  the  use  of  the  3ASS  programs,  and  an  appendix  with  instructions  for 
the  "WRITE,"  FWRITE,"  and  "5 WRITE"  programs.  These  latter  three  programs 
prepare  the  text  for  the  job  assessment  and  anchor  point  development  programs. 

INITIAL  USER  ACCEPTABILITY 

Preliminary  views  on  user  acceptance  of  the  3ASS  procedure  were  obtained 
at  FMC  Corporation,  San  3ose,  California  (largest  producer  of  military  tracked 
vehicles)  and  Northrop  Corporation,  Anaheim,  California  (producer  of  sensor 
units  for  the  OH-58D  Helicopter).  Based  on  these  preliminary  on-site  observa¬ 
tions  at  FMC  and  Northrop,  user  acceptance  of  the  3ASS  is  good.  ARI  and  MGA 
personnel  visited  these  sites  to  meet  with  potential  3ASS  users  from  the  human 
factors,  logistics,  training,  and  design  engineering  groups.  First,  a  presentation 
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of  the  overall  aptitude/task  tradeoff  procedures  that  occur  during  any  Army 
system  development  program  was  conducted.  This  presentation  began  with  the 
identification  of  the  need  for  a  technique  by  which  system  designers  can 
estimate  aptitude  requirements  of  operational  and  maintenance  personnel.  The 
presentation  then  traced  the  aptitude/task  trade  off  process,  identifying  the 
employment  of  job  assessment  techniques.  Next,  JASS  was  demonstrated. 
Finally,  comments  from  FMC  and  Northrop  personnel  were  solicited. 

Both  contractors  saw  the  utility  for  the  JASS.  They  agreed  that  it  would 
serve  as  means  of  communication  between  the  Army  and  system  designers.  In 
addition,  they  felt  that  it  would  serve  as  an  excellent  in-house  communication 
tool.  Hardware  design  engineers,  human  factors  and  training  psychologists,  and 
integrated  logistics  support  technicians  all  have  different  definitions  of  human 
abilities.  For  example,  a  maintenance  engineer,  human  factors  analyst,  and 
design  engineer  may  all  look  at  the  same  electrical  installation  drawing  but  will 
each  draw  different  conclusions  concerning  task  (and  therefore  aptitude)  require¬ 
ments.  The  human  factors  analyst  may  worry  that  a  person  wearing  Arctic 
mittens  will  not  be  able  to  repair  it;  the  maintenance  engineer  may  be  concerned 
with  the  mean  time  to  repair  it;  the  design  engineer  may  be  concerned  with 
manufacturing  tolerance  and  electrical  interference.  If  all  agree  on  a  common 
term  for  the  problem  (say,  high  demand  for  manual  dexterity),  it  will  help  to 
identify  and  eventually  correct  the  problem.  JASS  can  assist  in  making  design 
trade-off  decisions  based  on  a  framework  of  common  human  resource  demand 
definitions. 

FMC  and  Northrop  personnel  did  express  concern,  however,  about  the 
validity  and  comprehensiveness  of  the  40  psychologically  defined  aptitudes. 
Could  the  requirements  of  a  job,  such  as  an  electrician,  be  accurately  defined  in 


terms  of  these  40  aptitudes,  and  once  the  job  is  defined,  could  the  Army 
personnel  be  identified  and  selected  to  meet  these  requirements?  ARI/MGA 
personnel  explained  that  the  scales  had  already  been  used  and  found  adequate  to 
describe  some  military  jobs.  Further  testing  is  needed  to  ensure  reasonable 
coverage  of  the  full  range  of  military  jobs.  In  addition,  it  was  explained  that  the 
computer  itself  could  be  programmed  to  "learn"  job  aptitude  demands  and  assist 
in  the  definition  of  new  jobs.  The  Army  and  ARI  are  developing  programs 
whereby  military  personnel  can  be  tested  and  categorized  according  to  the 
aptitudes.  The  goal  is  a  common  definition  for  personnel  resources  and  job 
aptitude  requirements.  Some  hardware-oriented  engineering  personnel  distrust 
psychological  definitions  and  theories.  JASS  overcomes  this  problem  to  some 
extent  because  the  aptitudes  are  self  defining  in  non-psychological  terms  when 
one  follows  the  branching  network  flow.  Psychologically-based  definitions 
become  invisible  to  the  user. 

FUTURE  DIRECTIONS 

General 

The  basic  problems  concerning  development  of  a  computerized  job  assess¬ 
ment  system  have  been  resolved.  The  branching  network  job  assessment 
flowchart  was  implemented  in  software.  This  software  is  packaged  to  enable  a 
user  to  understand  and  successfully  operate  the  computer  system  with  minimal 
direction.  There  are  several  issues  to  be  resolved  and  capabilities  to  expand 
before  a  final  job  assessment  software  package  is  produced.  These  issues  tend  to 
fall  into  two  groups:  those  requiring  further  psychological  research  and  those 
requiring  modifications  to  the  system  software. 


User  Acceptance 

User  acceptance  is  of  primary  importance.  Validity  testing  or  any  program 
improvements  will  not  be  possible  unless  the  user  understands  and  feels  comfort¬ 
able  using  the  software  system.  3ASS  must  be  human  engineered  to  meet  the 
capabilities  and  limitations  of  a  potentially  wide  range  of  users.  Human 
engineering  and  user  acceptance  testing  have  already  been  done  in  a  very 
informal  manner  at  the  developers'  facilities  (see  Initial  User  Acceptability 
Section).  Further  testing  at  FMC  and  Northrop  Corporations  should  provide  a 
broad  enough  user  data  base  upon  which  some  programming  improvements  can  be 
made. 

Both  FMC  and  Northrop  were  extremely  interested  in  participating  in  pilot 
studies.  Each  company  proposed  that  a  small  group  of  their  engineers  rate  jobs 
on  systems  currently  under  development  within  the  company.  The  engineers 
would  be  selected  from  a  variety  of  disciplines:  human  factors,  logistics, 
training,  and  hardware  design.  Northrop  proposed  that  they  rate  the  job  of 
maintaining  the  mast  mounted  sight  for  the  OH-58D  Helicopter.  FMC  proposed 
several  alternative  hardware  items  including  the  DSWS  vehicle.  The  information 
to  be  obtained  from  these  studies  would  include: 

•  A  comparative  analysis  across  raters  which  would  give  some 
indication  of  the  model  validity. 

•  A  critique  of  the  task  examples  and  the  anchor  points  in  terms 
of  their  applicability  and  meaningfulness. 

•  An  analysis  of  JASS  results  compared  with  the  results  of  other 
currently  employed  job  assessment  techniques  (i.e.,  human 
factors  summaries,  QQPRI,  ILS  worksheets,  etc.) 

•  Subjective  comments  and  suggestions  from  test  subjects  regard¬ 


ing  system  usability. 


Anchor  Point  Specificity 


Aptitude  scale  anchor  points  could  range  in  scope  from  universal 
(applicable  to  any  job)  to  very  specific  (applicable  to  and  understood  by  only  one 
occupational  specialty).  Ideally,  there  exists  a  single  set  of  anchor  points  against 
which  to  weigh  any  job  ability  requirements.  This,  however,  will  rarely  occur. 
Tradeoffs  must  be  made  between  the  cost  of  specificity  and  the  amount  of 
information  which  is  sufficient  to  describe  the  aptitude  requirements  of  a 
weapon  system.  Anchor  point  specificity  can  be  addressed  through  both  usability 
and  validity  testing.  The  present  anchor  point  development  program  has  a  "skip" 
option  which  may  be  exercised  when  the  user  feels  the  task  is  not  appropriate  as 
an  anchor  point.  This  same  mechanism  could  be  built  into  the  job  assessment 
software.  The  user  could  be  asked  to  rate  the  anchor  points  in  addition  to  rating 
the  job  itself.  In  addition,  levels  of  specificity  could  be  established  and  anchor 
points  developed  for  each  level.  A  test  would  then  be  conducted  in  which 
subjects  rated  the  same  job  using  the  various  sets  of  test  anchor  points. 
Comparative  analyses  of  user  response  and  job  rating  results  will  assist  in 
determining  the  information  gained  by  increasing  specificity. 

Validity  of  the  3ASS  Procedure 

There  are  two  principal  types  of  validity  analysis  that  are  relevant  to 
3ASS.  Content  validity  is  established  through  a  logical  analysis  of  the  measure¬ 
ment  tool  to  determine  whether  it  measures  those  principles  it  is  supposed  to 
measure.  A  refined  3ASS  should  be  inherently  high  in  content  validity  since  it 
will  be  based  upon  a  careful  analysis  of  human  aptitude,  assessment  procedures, 
and  the  system  design  process.  An  additional  check  on  the  content  validity  of 
3  ASS  could  be  obtained  by  comparing  the  results  of  3 ASS  to  the  results  of  more 
traditional  job  analysis  techniques;  i.e.,  QQPRI,  task  analysis  summaries,- mainte- 
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nance  allocation,  etc.  This  method  would  have  the  advantage  that  the  research 
could  be  conducted  during  the  development  of  an  actual  weapon  system  (i.e., 
DSWS).  The  other  method  of  validation  that  should  be  investigated  for  JASS  is 
criterion  validity.  Criterion  validity  through  correlation  analyses  predicts  the 
degree  to  which  some  measurement  will  predict  a  specified  criterion.  In  the 
case  of  JASS,  this  criterion  would  be  the  actual  aptitude  required  to  operate  or 
maintain  a  fielded  system.  One  form  of  criterion  validity  testing  that  could  be 
done  with  JASS  would  be  historical  in  nature.  One  could  take  an  existing  Army 
system  (i.e.,  M109)  and  have  members  of  a  design  team  who  are  familiar  with  the 
system  rate  its  aptitude  requirements  using  JASS.  These  ratings  could  then  be 
correlated  with  measures  of  the  aptitudes  of  individuals  successfully  operating 
and  maintaining  the  fielded  system.  This  procedure  would  provide  a  direct 
comparison  between  predicted  and  actual  aptitudes  and  would  be  the  best 
measure  of  JASS'  criterion  validity.  However,  it  would  also  be  quite  demanding 
in  terms  of  time  and  troop  support.  A  more  economical  index  of  criterion 
validity  would  involve  system  designers  and  Army  subject  matter  experts  both 
using  JASS  to  analyze  the  aptitude  requirements  imposed  by  the  same  piece  of 
hardware.  This  analysis  would  reveal  how  well  the  estimates  of  human  aptitude 
requirements  made  by  the  system  design  community  will  match  those  made  by 
Army  personnel  who  are  familiar  with  the  actual  field  operation  and  mainte¬ 
nance  of  the  system. 

Program  Efficiency 

The  current  software  programs  are  written  in  the  BASIC  programming 
language.  While  BASIC  has  the  advantage  of  widespread  use  in  micro-computer 
systems,  it  is  difficult  to  modify  and  expensive  to  document.  BASIC  programs 
are  inherently  slow  and  the  user  must  frequently  wait  some  time  for  the  program 
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to  proceed.  Consideration  should  be  given  to  programming  the  system  in 
PASCAL.  This  would  simplify  program  revision  and  expansion  and,  therefore, 
the  programs  will  run  faster  and  more  efficiently.  BASIC  programming  language 
is  written  in  linear  format.  The  program  itself  does  not  operate  linearly, 
however.  Branching  subroutines  and  loops  are  all  hidden  within  sequentially 
numbered  lines.  PASCAL,  on  the  other  hand,  is  written  and  operates  in  a 
functional  format.  Functions,  variables,  and  constants  are  all  predefined  and 
grouped  in  a  straightforward  easy-to-read  format.  Program  revisions  necessary 
as  a  result  of  user  testing  will  be  more  easily  and  quickly  accomplished  in 
PASCAL. 

Increased  Computer  Capability 

At  its  present  stage  of  development,  3ASS  provides  a  neat,  compact,  and 
easy-to-use  job  assessment  tool.  Current  computer  technology,  however,  has  the 
potential  to  provide  a  much  more  powerful  tool.  For  example,  mini-computers 
have  considerable  more  storage  capacity  than  is  currently  used  for  3ASS.  There 
are  several  potential  sets  of  job  aptitude  rating  scales,  each  applicable  to  a 
different  career  field  (only  mechanic  and  helicopter  pilot  exist  now).  The 
computer  would  determine  the  appropriate  set  of  scales  for  a  particular  job 
assessment  through  a  set  of  introductory  questions.  An  introductory  program 
would  explain  the  use  of  the  system  and,  through  a  series  of  questions,  identify 
the  scale  set  most  appropriate  to  the  job. 

Artificial  Intelligence 

Current  programming  capabilities  make  it  possible  to  have  a  software 
system  that  "learns"  from  the  user  as  well  as  one  that  allowed  the  present  user 
to  learn  from  the  responses  of  previous  users.  Such  a  system  would  select 
questions  and  negotiate  responses  with  the  user  using  the  responses  of  previous 
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users  as  the  basis  of  negotiation.  The  programming  techniques  used  to 
accomplish  this  capability  are  called  Artificial  Intelligence  (AI).  The  following  is 
a  brief  example  of  how  this  might  work: 

The  system,  as  it  is  currently  envisioned,  would  be  menu  driven 
to  elicit  information  about  the  boundary  conditions  of  the  job  being 
assessed.  The  highest  order  menu  would  contain  global  job  titles  such 
as:  mechanic,  baker,  teacher,  pilot,  etc.  and  ask  the  user  to  select 
the  job  most  like  the  one  being  assessed.  If  the  user  selected  "1. 
Mechanic,”  the  next  menu  would  present  more  specific  information 
about  mechanics  such  as:  automotive  mechanic,  tracked  vehicle 
mechanic,  fixed  wing  reciprocating  engine  mechanic,  rotary  wing 
reciprocating  engine  mechanic,  etc.,  and  ask  the  user  to  select  that 
job  closest  to  the  job  being  assessed. 

The  final  menu  in  this  sequence  could  ask  the  user  to  select 
that  system  most  closely  resembling  that  in  which  the  job  incumbent 
will  be  working.  For  example,  if  the  user  has  selected  tracked 
vehicle  mechanic,  he  or  she  would  be  presented  with  a  menu  including 
systems  in  this  area.  An  example  menu  is:  Ml  13  APC,  M60A1  Battle 
Tank,  IFV,  Ml  Main  Battle  Tank,  etc. 

A  final  selection  at  this  level  would  initialize  the  scales  and 
data  matrices  with  the  boundary  conditions  for  a  mechanic  in  this 
weapon  system.  The  user  would  then  proceed  through  the  branching 
scales  "teaching"  the  software  how  the  new  system  requirements 
differ  from  the  old  system.  The  user  would  be  given  the  opportunity 
to  branch  around  inappropriate  scales  as  in  the  current  3ASS. 
However,  if  the  rater  skips  a  scale  which  had  been  rated  in  the  old 


system,  the  software  would  switch  mto  negotiation  mode  and  seek  to 
determine  what  accounts  for  the  system  differences.  This  would  also 
occur  when  the  user  rated  a  scale  especially  high  or  low  in  compar¬ 
ison  with  that  scale's  rating  on  the  old  system. 

The  development  of  this  software  would  seem  at  first  glance  to 
require  a  tremendous  amount  of  tedious  textual  input  which  would  be 
difficult  to  manage  and  produce.  This  is  overcome  by  the  nature  of 
A1  software.  A1  software  is  constructed  within  the  framework  of  a 
work  breakdown  structure  in  which  the  individual  functions,  proce  - 
dures  programs,  and  utility  programs  are  all  specified  in  a  formative 
(front  end)  analysis.  The  central  features  of  this  analysis  are 
twofold: 

1.  The  individual  blocks  of  the  system  each  perform  a  single 
and  non-redundant  function.  Block  control  is  always 
passed  off  to  the  executive  program  which  is  kept  at  the 
simplest  level  of  function,  i.e.,  it  does  not  input  infor¬ 
mation,  compute  outcomes,  or  output  information;  it 
simply  directs  the  execution  of  these  operations. 

2.  The  system  learns,  beginning  in  an  unbounded  state.  As 
users  input  increasing  system  level  information,  the  A1 
software  continues  to  reduce  uncertainty  about  new  sys¬ 
tems  as  well  as  systems  previously  entered. 

The  current  software  system  will  have  to  be  designed  and/or  programmed 
(e.g.,  in  PASCAL)  to  produce  the  AI  capabilities. 


Summary  and  Conclusions 


This  report  has  described  the  development  of  a  computerized  job  assess¬ 
ment  system  which  provides  a  profile  of  the  personnel  aptitudes  required  to 
operate  a  weapon  system.  The  system,  called  3ASS,  provides  a  convenient,  easy- 
to-use  method  for  designers  and  developers  to  describe  the  human  resources 
demanded  of  a  weapon  system.  Preliminary  testing  of  3ASS  was  described  and 
future  needs  discussed.  The  recommended  modifications  to  3ASS  are  expected 
to: 

•  Improve  user  acceptance 

•  Improve  validity  and  reliability 

•  Improve  programming  efficiency 

•  Determine  most  appropriate  types  of  anchor  points 

•  Develop  capability  for  software  to  "learn"  about  weapon  sys¬ 
tems  through  accumulated  user  input. 
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JOB  ASSESSMENT  SOFTWARE  SYSTEM  (JASS): 
USER'S  GUIDE 


SCOPE 

The  Job  Assessment  Software  System  (JASS)  is  a  set  of  tools  which  allows 
collection  of  job  assessment  data  using  a  microcomputer.  These  job  collection 
data  fall  into  two  categories: 

1.  Data  for  development  of  scale  anchor  points. 


2.  Job  assessment  data  collected  from  subject  matter  experts 
using  the  resultant  scale  anchor  points. 

This  guide  has  been  prepared  for  those  users  who  will  be  responsible  for 
managing  the  data  collection  for  both  the  development  of  behavioral  scale 
anchor  points  and  job  assessment.  The  programs  used  by  job  assessment  subject 
matter  experts  have  been  designed  to  be  free  standing  and  self  instructional. 


SYSTEM  CHARACTERISTICS  AND  REQUIREMENTS 


JASS  is  contained  on  5*  inch  floppy  discs.  The  system  requires  the  following 
hardware: 


•  Apple  II®  microcomputer  with  48K  memory. 


•  Video  monitor  (40  column,  color  or  non  color) 


Two  Apple®  disc  drives  with  disc  controller  mounted  in  slot  #6. 


•  Apple  DOS  3.3®  disc  operating  system 


Optional  hardware  includes  64K  memory  expansion  and  the  Appleu  game  paddles. 
All  of  the  programs  are  written  in  Applesoft®  Basic,  and  the  system  is  menu 
oriented  under  turnkey  operation.  This  means  that  the  system  is  "booted" 
(placed  in  use)  in  the  normal  manner. 


APPLE*®  REFERENCE  DOCUMENTATION 

If  you  are  unfamiliar  with  the  Apple®  microcomputer  system,  we  recommend 
that  you  read,  as  a  minimum,  Chapters  1  through  4,  and  Appendices  G  and  J  of 
Apple  II,  The  DOS  Manual.  The  programming  language  documentation  is 
contained  in  APPLESOFT  Q,  THE  BASIC  PROGRAMMING  REFERENCE  MANU¬ 


AL.  It  is  not  necessary  to  become  an  expert  programmer  in  order  to  use  JASS; 
however,  a  familiarity  with  these  reference  manuals  may  be  helpful  to  you. 


DISC  CARE  AND  HANDLING 

Floppy  discs  are  made  of  a  magnetic  recording  material  that  is  mounted  in  a 
permanently  sealed  square  cover.  DO  NOT  OPEN  THIS  PACKAGE.  This  will 


destroy  the  disc.  While  the  discs  are  called  "Floppies,"  they  can  actually  be 
destroyed  by  bending.  The  following  DOs  and  DON’Ts  should  be  routinely 
practiced. 


DO- 


Handle  discs  gently. 


Store  discs  in  the  paper  envelope  supplied  with  the  disc. 


Store  discs  vertically  in  a  cool  location. 


Back  up  (copy)  your  discs  routinely. 


Write  on  the  disc  label  with  a  ball  point  pen  or  other  sharp  writing 
instrument. 

Store  the  discs  where  you  suspect  magnetic  field,  i.e.,  on  the  top  of 
the  disc  drive  or  video  monitor,  or  near  permanent  magnets. 

While  the  definitive  answer  is  not  known,  the  present  authors  do  not  carry  discs 
through  magnetometer  inspection  stations  at  airports.  We  have  not  lost  one  yet! 

CAUTION 

NEVER  remove  a  disc  while  the  drive's  "IN  USE"  light  is  on.  This  may 
permanently  damage  the  disc,  and  is  almost  sure  to  destroy  the  information  on 
it.  In  such  a  case,  the  disc  can  usually  be  re-used,  but  you  won’t  be  able  to 
recover  the  lost  information. 

BACKING  UP  THE  SYSTEM  DISCS 

As  you  have  gathered  from  the  previous  section,  accidents  can  happen  and  discs 
can  "crash."  Now  is  a  good  time  to  back  up  your  program  to  insure  its  safety. 
You  should  establish  a  firm  practice  of  backing  up  discs  when  you  are  collecting 
data  to  insure  against  data  loss.  Copying  the  discs  can  be  readily  accomplished 
in  two  ways:  by  using  the  COPY  A  program,  or  the  FID  program  which  are  stored 
in  your  Apple®  System  Master  Disc.  To  use  COPYA,  follow  the  instructions  on 
page  38ff  of  the  DOS  Manual.  To  use  FID,  refer  to  page  184ff  of  the  manual. 
These  programs  have  slightly  different  characteristics,  and  you  should 
familiarize  yourself  with  them  before  attempting  to  copy  your  discs. 

SOFTWARE  SYSTEM  OVERVIEW 

The  software  contains  programs  which  develop  scaling  anchors,  collect  job 


assessment  data  using  the  scaling  anchors,  and  provide  summary  statistics  from 
several  people  assessing  a  job. 

JASS  is  divided  into  career  field  packages.  The  task  examples  in  each  package 
are  devoted  to  a  specific  career  field  (i.e.,  automotive  mechanic,  electrician, 
pilot,  etc.).  Two  career  field  packages  have  been  developed  to  date:  automotive 
mechanic  and  helicopter  crewman.  In  addition  to  a  difference  in  the  task 
examples,  there  is  a  slight  difference  in  the  programming  between  these  two 
packages.  The  anchor  points  in  the  job  assessment  portion  of  the  automotive 
mechanic  package  have  not  yet  been  scaled  and  are,  therefore,  at  four  equally 
spaced  points  on  the  scale  (high,  moderate,  some,  very  little).  The  anchor  points 
in  the  helicopter  package  have  been  scaled  and,  therefore,  vary  with  each  scale. 
The  programs  (FABD2  and  FABD2P)  are  slightly  different  to  handle  this 
variability  in  anchor  point  location.  Flexible  anchor  poir..  location  also  created  a 
need  for  "FWRITE,''  the  program  to  write  and  edit  these  flexible  anchor  points. 
(All  programs  begining  with  "F"  e.g.,  "FWRITE,"  "FABD2P,"  etc.,  are  designed  to 
handle  the  flexible  anchor  points.) 

The  automotive  mechanic  disc  package  consists  of  the  following: 

•  User's  Disc  -  Mechanic 

Contains  the  majority  of  system  programs.  Also  stores  the  job 
assessment  data  (stored  as  a  text  file  under  a  user-selected 
name)  and  anchor  point  development  data  (stored  as  text  files 
under  the  name  SCATA  XXXX,  where  XXXX  is  the  scale 
number). 

•  Disc  A-Mechanic,  Side  1  and  Side  2 

Contains  mechanic  job  assessment  transaction  and  scale  data 
block  text  files  under  the  name  DATA  XXXX  (where  XXX  is 
the  number  of  the  block).  Also  contains  WRITE,  a  program  used 
to  write  the  data  block  text  files. 


Disc  B-Mechanic 


Contains  automative  mechanic  anchor  point  development 
program  text  files  under  the  name  of  SC  XXXX  (where  XXXX 
is  the  number  of  the  scale).  Also  contains  SWRITE,  a  program 
used  to  write  the  anchor  point  program  text  files. 

The  programs  contained  in  the  JASS  Mechanic's  discs  are  functionally  shown  in 
Figure  1.  The  utility  programs  provide  the  computer  graphics  as  the  means  of 
writing  and  editing  the  text  files. 

The  helicopter  crewman  disc  package  consists  of  the  following: 

•  User's  Disc-Helicopter  Crewman 

Contains  a  majority  of  the  system  programs.  Also  stores  the 
data  for  the  job  assessment  data  (stored  as  text  files  under  a 
user-selected  name). 

•  Disc  A-Helicopter  Crewman,  Side  1  and  Side  2 

Contains  the  helicopter  job  assessment  transaction  and  scale 
data  block  text  files  under  the  name  DATA  XXXX.  Also 
contains  FWRITE,  a  program  used  to  write  the  data  block  text 
files. 

Disc  B  -  Helicopter  Crewman  is  still  in  development.  The  programs  contained  in 
JASS  Helicopter  Crewman's  discs  are  functionally  shown  in  Figure  2. 

In  addition  to  the  mechanic  and  helicopter  crewman  disc  packages,  a  mechanized 
infantryman  disc  package  has  been  started.  The  package  presently  has  just  one 
disc: 
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FIGURE  1:  JOB  ASSESSMENT  SOFTWARE  SYSTEM  (JASS)  -  MECHANIC 
(PERMANENT  ANCHOR  POINTS) 


UTILITY  PROGRAMS: 
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SWRITE 

HI-RES 

CHARTABLE 

Program  Number  as  Designated 
by  MENU 


TEXT 

FILES 


FIGURE  2:  JOB  ASSESSMENT  SOFTWARE  SYSTEM  (JASS)  -  HELICOPTER 
CREWMAN  (FLEXIBLE  ANCHOR  POINTS) 


Disc  B  -  Mechanized  Infantryman 


• 

Contains  mechanized  infantryman  anchor  point  development 
program  test  files.  This  disc  can  be  used  with  the  User's  Disc  - 
Mechanic.  Caution  must  be  taken,  however.  Do  not  use  the 
same  mechanic  User's  Disc  to  collect  both  mechanic  and 
mechanized  infantryman  anchor  data.  One  set  of  data  will 
overwrite  the  other.  Use  two  separate  mechanic  user's  discs  to 
collect  this  data. 

The  next  section  of  this  guide  describes  the  software  system's  individual 
components.  This  section  is  written  in  hands-on  interactive  form  and 
proceeds  quite  rapidly.  The  best  way  to  learn  what  each  component  of  the 
system  does  is  by  actually  setting  up  your  own  job  to  analyze,  and  then 
using  each  of  the  system's  components  to  perform  the  analysis.  We  have 
tried  to  make  each  program  self-explanatory  and  self-corrective.  If  you 
have  problems  or  criticisms,  we  would  appreciate  it  if  you  would  tell  us. 

SYSTEM  OPERATION 

When  the  User's  Disc  is  booted,  the  HELLO  program  brings  up  Applesoft  Basic, 
installs  the  high  resolution  character  generator  utilities,  and  runs  the  system 
controller  program  MENU  (or  FMENU).  MENU  presents  the  various  options  to 
the  user  and  passes  control  to  the  appropriate  transaction  program  based  upon 
the  user's  response.  When  the  transaction  is  completed,  that  program  passes 
control  back  to  MENU  for  the  next  option. 

Boot  the  User's  Disc  and  read  the  MENU.  Press  key  1  to  assess  a  job  and  read 
the  information  presented.  Press  "Y"  to  continue,  and  MENU  asks  whether  you 
want  to  use  the  game  paddles  or  the  keyboard  to  enter  information.  Depending 
on  the  response,  MENU  will  "RUN”  either  ABD2  (or  FABD2)  —  keyboard 
response,  or  ABD2P  (or  FABD2P)  —  game  paddle  responses.  Both  of  these 
programs  implement  the  branching  scales  network  for  job  assessment.  Run 


through  a  portion  of  the  Job  Assessment  program,  and  then  exercise  the  quit 
option.  Save  your  responses  under  your  name  and  when  you  return  to  MENU, 
select  option  2,  Review  Job  Assessment  (by  Rater). 

MENU  passes  control  to  the  program  CANAL  (or  FCANAL)  which  presents  you 
with  a  simple  graphic  review  of  your  ratings  from  highest  to  lowest.  Also,  you 
will  see  the  list  of  those  skills  not  selected.  Exercise  quit  option  and  return  to 
MENU. 

Now  select  option  3,  Revise  Job  Assessment.  The  first  part  of  the  REV  (or 
FREV)  program  is  identical  to  CANAL  except  that  a  pointer  appears  to  the  left 
of  the  list  of  skills.  Move  the  cursor  so  that  it  points  to  a  skill  rating  you  want 
to  change,  then  press  the  "RETURN"  key.  You  will  then  begin  the  job 
assessment  progam  (ABD2  or  FABD2)  at  the  specific  point  which  skill  rating  can 
be  changed.  All  skill  ratings  made  will  supercede  any  previously  made  ratings. 
Change  the  skill  rating  and  go  back  to  the  MENU.  - 

Job  Assessment  Tally  (TANAL  or  FTALLY),  MENU  option  4,  is  again  the  same  as 
the  CANAL  program,  except  that  the  skill  ratings  shown  are  the  averages  for 
several  users  (up  to  40).  The  program  also  computes  the  standard  deviation  of 
each  of  the  ratings.  Press  the  S(tatistics)  key  and  the  screen  will  display  the 
number  of  subjects  selecting  an  aptitude  scale,  the  mean  scale  score,  and  the 
standard  deviation.  Mean  and  standard  deviation  scores  are  based  on  a  scale 
from  0  to  7.  If  a  subject  "branches  around"  an  aptitude  scale,  the  scale  is  scored 
a  zero. 

As  with  the  Job  Assessment  programs,  there  are  two  scale  rating  programs, 
SCALED2  and  SCALED2P,  that  allow  the  user  to  rank  behavioral  anchors  on  the 
7-point  scale  using  either  the  keyboard  or  the  game  paddles.  If  the  user  is  not 
familiar  with  the  anchor,  or  feels  that  it-  does  not  apply  to  the  task,  then  the 
"SKIP"  selection  can  be  made.  In  this  case,  no  score  is  recorded.  These 
programs  are  accessed  through  the  MENU  option  5,  Develop  Scale  Anchor  Points. 

The  data  stored  by  the  two  s^ale  development  programs  are  accessed  through 
MENU  option  6,  Review  Anchor  Point  Data.  MENU  "RUNS"  program  RSC.  You 


will  be  required  to  enter  the  scale  number  and  the  task  number;  RSC  then 
displays  the  emulated  data  to  date  in  the  format  shown  in  Figure  3.  The  scale 
and  task  numbers  can  be  determined  from  Appendices  B  and  C.  Enter  scale 
number  1002  and  task  number  1  to  see  the  current  data  for  this  anchor  point.  To 
quit  this  program,  hit  the  return  key  twice. 


FILE  STRUCTURE  OVERVIEW 

The  User's  Disc  is  reserved  for  system  programs  and  user  response  data  files. 
Quit  the  MENU  (hit  key  7)  and  type  CATALOG  (return).  All  of  the  programs  are 
locked  as  indicated  by  an  asterisk  in  the  right  hand  margin.  In  order  to  protect 
the  programs  from  accidental  overwriting,  we  have  locked  them.  Since  the 
User's  Disc  is  used  to  store  response  data,  it  is  not  write-protected.  Unlocked 
entries  beginning  with  a  T  are  text  files  containing  user  response  data.  Text 
files  with  personal  names  are  job  assessment  response  data  files. 

Put  Disc  A  into  disc  drive  2  and  type  CATALOG,  D2  (return).  This  disc  contains 
text  file  names  DATA  nnnn,  each  of  these  files  is  a  "page"  from  the  Job 
Assessment  program.  These  text  files  are  shown  graphically  in  Appendix  B. 
There  are  two  flow  charts  in  Appendix  B:  one  for  mechanic  job  assessment  and 
one  for  helicopter  crewman  job  assessment.  Each  block  in  the  flow  chart 
represents  a  text  data  file.  The  number  in  the  block,  nnnn,  corresponds  to  the 
number  in  the  text  file  name,  DATA  nnnn.  These  files  can  be  created  or  edited 
using  the  WRITE  or  FWRITE  programs  on  the  disc.  Repeatedly  press  the  space 
bar  until  the  cursor  returns,  replace  Disc  A  with  Disc  B  and  type  CATALOG,  D2 
again.  Press  the  space  bar  until  the  cursor  returns.  You  will  see  files  named  SC 
nnnn.  These  are  the  scale  rating  files  that  contain  the  information  used  by  the 
anchor  point  scale  rating  program:  the  aptitude  definition  and  a  series  of 
example  tasks  (5  to  12)  that  the  user  is  asked  to  rate  as  aptitude  anchor  points. 
These  example  tasks  are  listed  in  Appendix  C  for  both  the  mechanic  and  the 
mechanized  infantryman.  These  files  can  be  created  and  edited  using  the 
SWRITE  program  on  this  disc.  See  Appendix  A  of  this  guide  for  instruction  on 
the  use  of  WRITE,  FWRITE,  AND  SWRITE. 


XXXX  XX  XXX/XXXX  XX  XXX/XXXX  XX  XXX  /  XXXX  XX  XXX/ 


SCALE  TASK  #  OF 
#  #  SUBJECTS 


XXX 


#  OF  SKIP 
RESPONSES 


(2x)Z 


NOTE:  x  represents  the  anchor 
point  scale 


FIGURE  3:  SCATA  XXXX  PROGRAM  DATA  COLLECTION  FORMAT 


eoHWflK 


USER  REFERENCE  SHEET 


MENU:  Enter  the  number  of  the  desired  selection,  i.e.,  "7"  to  quit. 

(1)  JOB  ASSESSMENT: 

•  Answer  all  questions  with  a  "Y"  or  "N." 

• '  Type  "Q"  to  quit. 

•  Type  "B"  to  backspace  and  review. 

•  GAME  PADDLE: 

-  Turn  pot  to  move  cursor. 

-  Push  the  button  to  store  your  response. 

•  KEYBOARD: 

-  Type  "U"  to  move  the  cursor  up;  type 
"D"  to  move  the  cursor  down;  and  press 
"RETURN"  to  indicate  your  response. 

(2)  JOB  ASSESSMENT  REVIEW  (BY  RATER): 

•  Press  space  bar  to  review  all  ratings. 

•  Type  "Q"  to  quit. 

(3)  REVISE  JOB  ASSESSMENT: 

•  Press  space  bar  to  review  all  ratings. 

•  Type  "U"  to  move  the  cursor  up. 
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•  Type  "D"  to  move  the  cursor  down. 


•  Press  "RETURN"  to  indicate  selection  of  rating  to  be  revised. 

•  Type  "Q"  to  quit. 

(4)  JOB  ASSESSMENT  REVIEW  (TALLY): 

•  Press  space  bar  to  review  all  ratings. 

•  Type  "S"  to  review  rating  statistics. 

•  Type  "Q"  to  quit. 

(5)  SCALE  ANCHOR  POINT  DEVELOPMENT: 

•  Type  "Q"  to  quit  when  skill  description  is  on  screen. 

•  Type  "B"  to  backspace  to  skill  description. 

•  GAME  PADDLE: 

-  Turn  pot  to  move  arrow  cursor. 

Push  the  button  to  store  your  response. 

•  KEYBOARD: 

-  Type  "U"  to  move  the  cursor  up;  type 
"D"  to  move  the  cursor  down;  and  press 
"RETURN"  to  record  your  response. 

(6)  ANCHOR  POINT  DATA  REVIEW: 


Enter  task  number  and  press  "return"  key. 


•  For  different  task  on  same  scale,  enter  task  number  and  press 
"return"  key. 

•  For  different  scale,  press  "return"  key  (without  entering  task  number) 
and  then  enter  new  scale  number. 

•  To  quit,  press  "return"  without  entering  new  scale  number. 


APPENDICES 


APPENDIX  A: 

APPENDIX  B: 

APPENDIX  C: 


DSE  OF  PROGRAMS  WRITE,  SWRITE,  AND  FWRITE 
JOB  ASSESSMENT  FLOWCHARTS 

•  MECHANIC  JOB  ASSESSMENT  FLOWCHART 

•  HELICOPTER  CREWMAN  ASSESSMENT  FLOWCHART 

ANCHOR  POINT  TASKS 

•  MECHANIC  ANCHOR  POINT  TASKS 

•  MECHANIZED  INFANTRYMAN  ANCHOR 
POINT  TASKS 


USING  THE  WRITE  PROGRAM 


Program  will  RUN  and  print 
out  a  short  explanation  at  the 
top  of  the  screen,  then  BEEP, 
and  print  "BREAK  IN  1." 


2.  Type  in  Data  Statements  (starting 
with  line  100)  in  the  following 
format: 

100  Data  " . " 


2.1  If  you  are  typing  a  TRANS¬ 
ACTION  BLOCK  (yes/no 
response)  the  entire  text  will  be 
in  line  100. 


TO  WRITE  A  NEW  TEXT  FILE: 

1.  Type  1  RUN  WRlfSl  and 
^Return^ 


USING  THE  WRITE  PROGRAM 


USING  THE  F WRITE  PROGRAM 


TO  WRITE  A  NEW  TEXT  FILE: 


1.  Type  j  RUN  FWRITE 
<Tteturn^> 


and 


Program  will  RUN  and  print 
out  a  short  explanation  at  the 
top  of  the  screen,  then  BEEP, 
and  print  "BREAK  IN  1." 


2.  Type  in  Data  Statements  (starting 
with  line  100)  in  the  following 
format: 

100  Data 


2.1  If  you  are  typing  a  TRANS¬ 
ACTION  BLOCK  (yes/no 
response)  the  entire  text  of  the 
statement  will  be  in  line  100. 


2.2  If  you  are  typing  a  SCALE  DATA 
BLOCK,  type  Data  lines  in  the 
following  order  (where  x.xx  is  the 
task  score  from  0  to  7) 

100  Data  "x.xx..."  (task  with 
highest  score 

105  Data  "x.xx..."  (task  with 
lowest  score) 

•  110  Data  "xx.xx..."  (if  there  are  3 
tasks) 

115  Data  ".  .  .Scale  Title.  . ." 


3.  Type 


GOTO  2 


and  ^Return^ 


Screen  will  clear  and  then  ask 
for  the  filename. 


4. 


Type  the  data  block  number 

and  ^Return^ 


.4  Digits. 


Filename  prints  out  at  top  of 
screen  and  question  prints  out 
"EDIT  ONLY?" 


5.  Type  j  N  |  and  <^Return^> 


Asks  a  series  of  questions  a- 
bout  block  number,  position, 
type. 


6.  Respond  to  questions  and  ^Return^ 


Screen  clears  when  the  TEXT 
FILE  has  been  written. 


USING  THE  F  WRITE  PROGRAM 


TO  EDIT  AN  EXISTING  TEXT  FILE: 


1. 


Type 
<(Return/ 


RUN  FWRITE 


and 


Program  runs  and  prints  out  a 
short  explanation  at  the  top  of 
the  screen  then  BEEPS,  and 
prints  "BREAK  IN  1." 


2. 


Type  |  EXEC  DATA.  .  . 
^Return^ 


and 


Series  of  cursors  will  scroll  up 
on  screen  and  may  BEEP  and 
print  out  "UNDEF’D 
STATEMENT  ERROR  IN  350." 
Pay  no  attention  to  this  error 
message. 


3.  Type  [  LIST  ]  and  ^Return^> 


The  screen  will  show  the 
FWRITE  Program  (up  to  line 
90)  and  the  TEXT  FILE  (from 
line  100  on). 


4.  Edit  the  Text  File  Lines  as  neces¬ 
sary 


5.  Type  LIST  and  ^Return^  to 


make  final  check 


6.  Type  GOTO  2  and  <^Return^ 


The  screen  will  clear  and  then 
ask  for  the  Filename. 


Type  in  block  number 


.  .  .4  digit.  .  .  (new  one  if 


you  wish  or  the  old  number)  and 
<[Return)> 

8.  Type  |  Y ]  and  <^Return^> 


Filename  prints  out  at  top  of 
screen  and  asks  "EDIT 
ONLY?" 


Asks  if  the  block  follows  a 
Scale  Data  Block. 


9.  Answer  question  and  ^return  )> 


Screen  clears  when  the  Text 
File  has  been  edited. 


USING  THE  S WRITE  PROGRAM 


TO  WRITE  A  NEW  TEXT  FILE. 


Type 


RUN  SWRITE 


and  <^Return^ 


Program  will  RUN  and  print 
out  a  short  explanation  at  the 
top  of  the  screen,  then  BEEP, 
and  print  "BREAK  IN  1." 


Type  in  Data  Statements  (starting 
with  line  100)  in  the  following  for¬ 
mat: 

100  Data  "...SCALE  TITLE..." 

105  Data  "...SCALE  EXPLA¬ 
NATION..." 

110  Data". ..TASK  1  STATEMENT...", 
"....TAS'  2  STATEMENT...." 


Note:  If  the  computer  beeps 
in  the  middle  of  the  Scale  Ex¬ 
planation,  this  means  that 
there  too  many  letters  in  the 
line.  Shorten  the  explanation 
to  fit  within  the  line.  If  the 
computer  beeps  in  the  middle 
of  a  task  statement,  erase  the 
portion  started  and  start  over 
on  the  next  line. 


199  Data  "..TASK  N  STATEMENT.." 


If  this  is  the  first  scale,  then  type 
an  additional  line:  90  REM 


Screen  will  clear  and  then  ask 
for  the  scale  number. 


Type  GOTO  2  and  ^Return^ 


Filename  prints  out  a  top  of 
screen  and  question  prints  out 
"EDIT  ONLY?" 


Type  Scale  Number 
and  ^Return  ^ 


....4  Digits, 


Asks  for  the  number  of  tasks 
and  the  next  scale  number. 


Type  N  and  <^Return^ 


Respond  to  Questions 


and  <^Return^ 


Screen  clears  when  the  text 
files  have  been  written. 


USING  THE  S WRITE  PROGRAM 


TO  EDIT  AN  EXISTING  TEXT  FILE: 


Program  runs  and  prints  out  a 
short  explanation  at  the  top  of 
the  screen,  then  BEEPS,  and 
prints  "BREAK  IN  1." 


Series  of  cursors  will  scroll  up 
on  screen,  a  BEEP  sounds  and 
then  the  screen  goes  back  the 
way  it  was. 


3.  Type  [LIST  [  and  ^Return^> 


The  screen  will  show  the 
SWRITE  Program  (up  to  line 
87)  and  the  TEXT  FILE  (from 
line  100  on). 


4.  Edit  the  Text  File  Lines  (lines  100 
to  199)  as  necessary 

5.  Type  1  LIST  |  and  ^Return^to 
make  final  check 


6.  Type  |  GOTO  2  |  and  <fReturn^  The  screen  will  clear  and  then 

ask  for  the  Filename. 


7.  Type  in  Scale  Number  (new  one  if  Filename  prints  out  at  top  of 

you  wish  or  the  old  name)  screen  and  asks  "EDIT 

....4  Digits....  and  <^Return}>  ONLY?" 


,  8.  Type  [  Y  j  and  <^Return^> 


Screen  clears  when  the  Text 
File  has  been  edited. 


APPENDIX  B 

JOB  ASSESSMENT  FLOWCHART 

There  are  two  branching  network  flowcharts  in  this  appendix.  One  shows  text 
data  files  for  mechanic  job  assessment  and  the  other  shows  the  text  data  files 
for  helicopter  crewman  job  assessment.  These  text  data  files  are  found  on  Disc 
A.  A  separate  text  data  file  exists  for  each  data  block.  They  are  titled  DATA 
XXXX,  where  XXXX  is  the  number  at  the  top  of  each  data  block.  Each  text 
data  file  contains  the  full  text  that  is  shown  on  the  screen  plus  flags  for  the  next 
text  data  file. 

The  branching  network  data  files  are  shown  here  for  three  purposes. 

•  Modifications  can  be  made  to  the  branching  network  using  WRITE 
(for  mechanic  job)  or  FWRITE  (for  helicopter  crewman)  programs. 
The  data  block  numbers  are  required  to  make  these  modifications. 

•  The  scale  data  block  numbers  are  required  when  analyzing  the  results 
of  the  anchor  point  development  program  (using  RSC  program). 

•  A  knowledge  of  the  branching  network  flowchart  may  be  helpful  when 
using  the  backspace  feature  in  job  assessment  or  the  job  assessment 
revision  programs  (FREV  or  REV). 


B-  1 


c  oc 
01  c 

•u  Ld 
(0 

•w  c 

H  V 

o  c 

—t  d  G. 

O  cn 

o 

■H  d  T3 
18  C 
0)  (3 
0}  u 

(U  in  r-. 
O  d  05 

!  o)  0)  oj 
c  -a  u 
c  c 

*J  3  t) 


C/3 

z  z 

-y  cd  Id 

OH  X 

OH  Id 

. — r  <— I  Cd 

cd  H 

3  Z 

O 

o 


H 

H 

x 

Gb 

£ 

c 

3 

0 

O 

CO 

•H 

w 

XJ 

x 

0) 

to 

XJ 

X 

E  ' 

XJ 

U  i 

w. 

o 

o  \ 

XJ 

| 

CC 

XJ 

C  1 

*— < 

U 

eC 

•H 

•H 

jZ 

O 

C 

XJ 

CO 

rW 

3 

| 

E 

X 

£ 

O 

O 

CO 

T3 

o 

C 

6C 

C9 

0) 

c 

XJ 

x 

•w 

CO 

tJ 

XJ 

to 

<u 

1 

0) 

o 

3 

Cl 

e 

s: 

CO 

3 

«.-i  c 
3  -H  O 
O  3  -H 
d  d 

x  m  is 
d  d  E 
03  d 
OXO 

■LJ  _J  U- 

o  o  c 

U  -H 


E  d  X 
E 

o  c  -a 
urns 
c 

o  ec  jj 

X  C  CC 
-d  d 
u  u  3 
K  *H  *a 
3  d  C 
Z  3  3 


>* 

H 

>4 

M 

fa 

►J 

m 

o 

M 

m 

o 

« 

o 

fa) 

M 

—■ H 

H 

X 

< 

u: 

O 

♦-3 

C&4 

z 

o 

w  U 

H  Z 


o 

t 

U-4 

u 

*H 

*3 

S 

o 

C 

-C 

•H 

rH 

CO 

rH 

CiO 

0) 

c 

■U 

•H 

-C 

<r 

CO 

4-J 

m 

CD 

o 

rH 

U-| 

*— ♦ 

3 

O 

H 

4J 

col 

CD 

0) 

CO 

CO 

CO 

0) 

£ 

-C 

(0 

4-1 

4J 

cx 

c 

■u 

3 

0) 

CO 

o 

H 

3 

H 

0) 

X 

00  IM 

O  S 

o  2 

r-l  O 
fa 

z 

M 


o 

u 

z 

(-< 

M 

CN 

H 

z 

< 

o 

O 

2 

w 

*-h 

fa) 

< 

M- 

fa) 

H 

fa 

< 

2 

o 

±J  4J 

^  -a 

X  CD  <U 
O  -H  > 


^  O  C" 

v  u  n 
.c  vj 

AJ  W  4) 

a)  3 
a)  to  a: 

rt  C 
<  u  a 


<4-1 

U  -H 

o  o 

QJ 

V4  Cl. 

QJ  CO 

•o 

V4  'CO 
_  O 
ri  c 
LO  c  -h 

O-  4J 

—  to 
•o  OC 
CD  C 
CO  -H 

3  X  1 

4J 

^  I 

QJ  0) 
j:  oo  c- 
u  a  u 
CO  0) 
CD  W  T3 
W  U  M 
<  rt 


i 

*H 

1 

1 

J.J 

CO 

4J 

00 

CO 

rH 

o 

u 

£ 

3 

c 

o 

QJ 

a 

i-h 

QJ 

u 

4J 

CO 

U 

o 

CO 

o 

CO 

E 

c 

*H 

CO 

00 

00 

CO 

c 

•H 

c 

3 

o 

CO 

•H 

AJ 

•H 

01 

CO 

o 

4-1 

*— i 

33 

3 

< 

CO 

'w' 

4-1 

O 

c 

E 

3 

rH 

o 

QJ 

a 

CO 

»— < 

E 

u 

CO 

O 

0) 

O 

4-1 

u 

a 

u 

a 

Ou 

QJ 

*3 

QJ 

o 

C 

• 3 

•c 

CO 

c 

CO 

0) 

4-1 

o 

U 

0) 

o 

CO 

-H 

4J 

M 

c 

3 

CO 

•H 

rH 

o 

3* 

3 

C 

CO 

*H 

QJ 

X 

<0 

o 

4J 

Li 

P> 

rH 

a. 

CO¬ 

CO 

Li 

O 

c 

O 

c 

CO 

vO 

•  CO 

a 

O 

w 

rH 

— < 

a 

3  e- 

a 

H  CO 

a 

3 

00  r-! 

J= 

c  cx 

4J 

•H  *H 
4J  U 

4-1 

CO  c 

CO 

•H  *H 

3 

X 

E 

O .  C- 

c 

3 

3 

o 

*H 

CJ 

H 

> 

3 

E 

o 

p 

C 

c 

•H 

H 

o 

c 

r-* 

-X 

O 

CO 

rH 

*— < 

3 

3 

4J 

U 

•H 

c^. 

3 

4-1 

CO 

r* 

3 

4-1 

4J 

E 

a 

O 

a 

CO 

sz 

u 

o 

4-1 

r* 

c 

3 

C 

c 

E 

u 

m  . . 

#' 


r?7777rr?7y7?7Tr*7?  r?n 


UJ 


p:;;: 


m 


05 

N 

«H 

e 

o 

05 

"H 

X 

AJ 

« 

«u 

£ 

«3 

o 

3 

C 

3 

*H 

O 

c 

a) 

CM 

J* 

rH 

Ij 

o 

c 

CM 

0 

3 

05 

•H 

05 

V- 

a 

O 

Vi- 

at 

r- 

GO 

ij 

C 

•H 

05 

05 

c 

O 

o 

o 

“ 

c 

c 

u 

*H 

Cl 

■U 

s 

AJ 

o 

C3 

c 

a 

m 

-* 

»H 

a 

o 

c 

JC 

CM 

o 

AJ 

05 

u 

AJ 

05 

tc 

a  jc 

i 

05 

C^* 

JZZ 

05 

05 

■u 

U 

-3 

c 

05 

13 

0) 

> 

rH 

o 

”3 

•H 

a 

3 

»H  I 
M  GO 
U  I  c 

«H  l-i  «H 

3  O  C 
cr  vw  03 
C  QJ 
C  -H  E 
O 

(0  ft)  (3 
P 

<T  05  a-J  O  C 

•na  aj  v, 
o  05  C  05 

N  fl)  P  -H  U 

X  3  *j 

4-i  4-»  c  c 
o  o  a 

4J  3  *H 

05  l-«  4-»  »H 

3  ij  r,  : 

£  (I!  EU4 


m  O  »H 

u  »-  a 
o  <3  3 
c  « 

£iJS  U 
H  U  Q 
•H 

C  £  O 

O  5  J2 

_  W  ^ 
o^tt 
CM  ©  C 

o  a  o  o 

CM  «H 
ai  iJ  u 
x  o  u 
a->  CJ  «S 
u  a  e- 

AJ  A-l  ^ 

oj  a:  aj  to 
3  *H  0  <3 

S  T3  C  ti 


TO  202 


■  WV  /  ■ 


U 


w 

> 

o 

rH 

*— < 

*— < 

Csl 

H 

H 

c 

O 

Z 

rsi 

w 

U 

J 

H 

U3 

H 

C/3 

< 

rv 

j 

< 

E 

Q 

IsJ 

co 

ft- 

W 

o 

w 

C- 

CN1 

u 

CO 

• 

cd 

w 

ft- 

0) 

•o 

S3 

£ 

<y 

c- 

X) 

>> 

rH  1 

03 

y  1 

C 

4-i 

o 

C3 

CO 

U 

ro 

-iH 

3 

O 

V- 

y 

CM 

« 

y 

& 

d 

E 

O 

T3 

u 

c 

d 

0) 

X 

>* 

*J 

rH 

u 

y 

03 

•H 

3 

3 

£ 

a* 

o 

*j  e^ 

±J 

-H  X  ' 

>% 

5  co 

C  I 

I-* 

X  -4  ! 

c"> 

c 

eox  1 

m 

X 

C  4-» 

o 

X 

•fH 

CM 

a 

X  "O 

u 

4J  Ci 

y 

y 

e 

a  o 

jj 

w»  X 

C  E 

H 

C-  CJ 

X 

£  £ 
o  o 

y  v- 

JDVOZ  or 


I*. 


H 

< 

iH  M 

m  m 

O  i-J 

CM  < 

3 
t A 


0 

CO 

x: 

AJ 

X 

0 

5 

aj 

o 

c 

C 

C- 

0 

CO 

•H 

AJ 

c 

AJ 

u 

r— ( 

o 

CO 

-3“ 

CO 

iH 

•n 

O 

(l) 

X> 

CN 

0 

O 

a 

C 

CM 

0 

•H 

o 

x: 

AJ 

C 

e 

o 

o 

*0 

•H 

*H 

rH 

AJ 

4J 

3 

CO 

CO 

o 

a 

a 

x: 

o 

o 

C/3 

c 

o 

•H 

- 

AJ 

CO 

cu 

tH 

x 

0) 

u 

u 

S 

c 

o 

•H 

c 

o~ 

.s! 

CO 

c 

AJ 

o 

c 

u 

ro 

o 

0 

AJ 

-3- 

cn 

*«-> 

CO 

O 

Vj 

a 

CM 

tu 

o 

o 

a. 

?H 

uj 

cu 

O 

P 

X 

5 

XJ 

c 

o 

o 

•a 

•H 

CO 

r— 1 

AJ 

•H 

D 

CO 

O 

u 

X 

o 

o 

w 

1— 1 

AJ 

1 

O 

c*» 

o 

0 

CO 

|P 

CO 

AJ 

AJ 

AJ 

3 

o 

c 

0 

XJ 

x= 

CO 

AJ 

AJ 

c 

C 

CM 

o 

o 

CO 

o 

MT 

*H 

AJ 

O 

AJ 

u 

0 

CN 

CO 

0 

u 

6 

a. 

c 

u 

B 

CO 

0 

*H 

£ 

U_i 

j-i 

c 

C 

0 

•H 

0 

U-i 

•H 

M 

CO 

AJ 

0 

M 

c 

a. 

AJ 

£ 

i 

•H 

C 

w 

0 

CO 

^Z 

o 

E 

AJ 

AJ 

AJ 

U-j 

O 

•H 

»J 

CO 

CO 

c** 

0) 

O 

U 

a  ”0 

Jj 

AJ 

5 

0 

0 

.  -H 

O 

AJ 

CO 

tc 

0 

o 

AJ 

c 

0*  rH 

CO 

*H 

0 

0 

XJ 

U-l 

u 

o  >- 

co 

O 

u 

lPi 

o 

O 

0 

C 

a 

CO 

o 

CN  CO 

0 

f-H 

a 

CO 

cc 

z 

u 

AJ 

cO 

r— 

3 

c 

ff 

O 

c 

a 

CO 

•M 

•M 

u 

a 

u 

0 

a 

M 

a 

X3 

to 

*“ * 

AJ 

“0 

a 

a 

a 

r- 

-H 

> 

> 

o 

5 

XI 

c 

0 

c 

AJ 

E 

AJ 

£ 

x: 

AJ 

c 

0 

0 

o 

a 

»— i 

AJ 

•H 

o 

VM 

CO 

0 

■H 

3 

Jj 

C 

£ 

t-t 

co 

3 

•H 

H 

O’ 

CO 

0 

a 

u 

O 

w 

0 

•H 

o 

CO 

o 

o 

>> 

m 

CO 

CO 

0 

3 

c. 

AJ 

c 

U-l 

c— 

o 

AJ 

C 

JZ 

x^ 

AJ 

AJ 

z 

AJ 

CC 

0 

■  3 

c 

CO 

CO 

3 

o 

a 

Sj 

O 

V- 

c 

o 

E 

AJ 

a. 

0 

CO 

H 

»-h 

H 

fJ 

rH 

Z 

>-H 

w 

O 

H 

h-i 

cn 

X 

X 

w 

w 

*x 

fc-< 

>< 

§ 

Q 

M 

O 

ctf 

rH 

« 

« 

rn 

l-l 

o 

V) 

-! 

n 

cn 

M 

o 

3 

« 

O' 

o 

td 

H 

o 

Z 

CM 

P3 

(d 

<r 

Cl 

s 

s 

o 

Cd 

t-t 

Id 

m 

Id 

id 

> 

eu 

O 

w 

s 

0) 

<D 

XI 

rH 

r 

u 

1 

r3 

4J 

c-* 

1 

3 

CO 

CO 

04 

0) 

*H 

a 

GC 

rH 

04 

•C 

04 

•H 

X> 

-O 

4J 

4J 

«-H 

3 

*H 

cr 

X 

4J 

r 

J= 

J-4 

0) 

04 

CO 

x: 

4-1 

0 

p 

rH 

3 

a 

•H 

O 

U-. 

e 

xJ 

5 

"3 

04 

04 

C 

1  O 

co 

<D 

04 

M 

4-1 

3 

1  m 

3 

-O 

XT 

4-> 

3 

4J 

CO 

0 

co 

O 

XJ 

6 

04 

4-1 

3 

r 

xr 

u 

X 

.d 

”3 

u 

CO 

4J 

c 

u 

3 

3 

o 

a 

04 

* 

co 

co 

XU 

X> 

U 

>S 

04 

p 

o 

*3 

o 

04 

3 

o 

U 

O 

Q 

O- 

CO 

4-> 

O 

X2 

0)  C 

u 

04  *H 

04 

c 

d<! 

JZ 

r* 

<U 

4J 

4-4 

O  V 
4J  C 

E 

>» 

3 

H 

O 

U 

a  <-* 

O 

m 

3 

rH  3 

U-l 

o 

CO 

3  d) 

U 

m 

CO 

3 

04 

g; 

CO 

CL 

u 

Q)  *H 

ei 

£>  d: 

o 

c 

c  c 

U 

4-4 

O  *H 

U 

*H 

CO  3 

04 

U  M3 

CO 

CJ  <u 

HH 

C.  d 

O 

to 

CO 

*H 

E 

w 

r 

C4 

3 

r; 

CO 

4J 

CO 

3 

iH 

XJ 

U 

0 

o 

a 

u- 

04 

<r 

XI 

> 

o 

4J 

>> 

O 

m 

u 

e 

£ 

3 

V* 

CO 

o 

O 

CO 

4-4 

c- 

c-i 

a 

CO 

ia 

u 

c 

CO 

04 

o 

o 

a; 

3. 

!■*■ 

CO 

rH 

w 

0 

4J 

3 

u 

H 

•H 

a. 

o 

H 

aC 

a 

w 

tJ 

w 

CM 

hH 

O 

w 

fH 

OS 

z 

Cu 

O 

<r 

3 

M 

CO 

co  o*-« 

sz  u 

H  tl  CO 
rH  CO 
O  t»  VH 

<3*  C 
CO  0) 
4J  'O 
V-  CO 
O  £ 

a. 

6  a> 
•h  x> 


a  w  w 

Jj  -W  -H 

sz  u 

s  * 

U-i 

CO  o  u 


O  o  C  t/5 
f— i  CC  o  !-■ 

o  u  s  o 
<r  u  o  c£ 
C-  >  c 
O  *H 

a  e 

4J  *0  - 

a  w 

u  ti  -o 

w  co  c 
3  a  co 
oc  cuj: 


.v/.y 


'0  4030 


z 

J  o 

O  M 

n  a!  to 

CO  H  i-l 

o  z  o 

<J-  O  Id 

U  OS 

CU 


13 

0> 

4—1 

0) 

3 

o 

-*—> 

■w 

13 

CO 

>> 

H 

0) 

13 

X) 

0> 

rH 

4J 

cn 

cc 

cC 

O 

0)  c^- 

'O’ 

o 

O.  CO 

u 

<u  c 

4-1 

H  o 

c 

■H 

o 

13  4-> 

u 

C  -H 

CO  CO 

0) 

O 

>*  Q. 

•u 

»-4 

4J 

4J 

u  o 

CO 

*H  c3 

3 

3  X 

X 

O*  0) 

13 

V 

4J 

>N 

03 

H 

3 

03 

•*1 

3 

13 

O 

e- 

(0 

3 

03 

0 

C 

C 

0) 

•H 

03 

X) 

u 

4_> 

O 

0) 

e 

03 

m 

CO 

V 

O 

• 

»— 4 

Cl 

u 

V4 

rn 

o 

(0 

o 

o 

Ui 

CQ 

4-1 

c 

4J 

c 

U-l 

U 

o 

o 

0) 

u 

CO 

**“> 

0) 

c 

JZ 

o 

00 

o 

o 

JZ 

c 

4J 

2 

4-1 

to 

u 

c 

4J 

*<3 

o 

•H 

CD 

u 

> 

3 

o 

«H 

o 

X 

4J 

u 

6 

a>  o  «>- 

u  l-i  Cl 

—  i-H 

SCO 

tr  o  -h 
oi  u  x 


M-l 

O  to  O  M 
CO  GJ  0 
O  J-J 
<■  CO 

o  o  c 
x  e  -h 

U  U  £ 

05  U 

C8  s  <c 
O  —i  E 


Em 


V 


'TO  403-4 


HELICOPTER  CREWMAN  ASSESSMENT  FLOWCHART 


2 

o 

l-l 

CO 

2 

2 

cn 

< 

2 

o 

00 

2 

o 

os 

2 

rH 

2 

2 

> 

2 

z 

o 

O 

Bl 


CO 

2  2 

•»  H  2 

OH  2 

OH  2 

.—I  H-l  2 

2  H 

3  s 

u 


2 

O 

2 

*-H 

r-> 

< 

GO 

H 

a 

CO 

O 

2 

w 

rH 

w 

ftS 

> 

a. 

X 

w 

2 

2 

O 

2 

M 

<■ 

H 

CO 

•-H 

H 

CO 

o 

►H 

u 

iH 

2 

rv» 

3 

CH 

X 

2 

TJ 

c 

2 

CO 

co 

*H 

CO 

T5 

rH 

2 

CO 

60 

P- 

<u 

c 

CO 

2 

u 

60 

o 

e 

CO 

Cs» 

AJ 

y 

H 

o 

AJ 

CO 

o 

>%  AJ 

D. 

I— I 

V 

*H 

to 

V 

~o 

CO 

3 

c 

co 

CO 

y 

2 

V 

C 

CO 

aj 

(0 

g 

c 

AJ 

a 

CO 

c 

AJ 

H 

y 

*H 

(1) 

AJ 

*3 

c 

CO 

c 

0) 

M 

3 

CO 

60  »  C 
3  O 
o  a  rl 

1-1  I*  XJ 

2  0)  TO 
•u  2  E 

jj  ^ 

0)  o  o 

XJ  CM 

SUE 
H  U  E  tl 
■— I  fl  2 
O  C  AJ  » 
H  3  -H 

§2 

o 

OKU 

y  c 

60  RJ 
0)  C  XJ 
2  i-l  ID 
.as  ij 
•u  to  y 

03  0)  T3 
3  C.  C 
Z  ro  3 


r* 

rH 

e*- 

60  rH 

c 

3 

•H 

o 

o 

3 

•H 

V» 

AJ 

r 

CO 

CO 

AJ 

Vi 

E 

a) 

Vi 

y 

2 

O 

AJ 

AJ 

lAl 

re 

o 

c 

u 

•H 

CM 

*H 

AJ 

rH 

c 

CO 

CO 

o 

3 

r* 

*H 

rH 

t= 

E 

O 

AJ 

— 

o 

•g 

u 

CO 

c 

CO 

G 

60  AJ  1 

r“ 

c 

CO 

•H 

Vi 

AJ 

AJ 

y 

CO 

•H 

x> 

3 

Vi 

c 

2 

3 

3 

c 

o 

CO 

Vi 

AJ 

a 

3 

c- 

O 

a 

Si 

o 

AJ 

o 

CO 

*H 

CO 

O 

r“ 

a. 

O 

AJ 

AJ 

a 

o 

■H 

rH 

AJ 

C 

a 

AJ 

J3 

60 

C 

CO 

v> 

I 

*3 

60 

o 

E 

g 

AJ 

■3 

O 

rH 

o 

a 

AJ 

G 

rH 

60  rH 

u 

o 

o 

c 

> 

G 

g 

c 

C-* 

rH 

*H 

o 

r- 

10 

E 

c 

V> 

3 

3 

AJ 

AJ 

G 

CO 

CO 

CO 

CO 

CO 

CO 

JZ 

AJ 

< 

•rH 

CO 

E 

AJ 

o 

/  CM 

o  2 
o 

V  o 


2000  2001  2002 
Does  the  sensory  informa-  Must  the  person  switch  back  DIVIDED 

tion  which  must  be  used  - ►  and  forth  between  the  tw:  or  - 

in  the  task  come  from  two  YES  more  sources  of  information  YES  ATTENTION 


TO  20 1 A\  /TO  2012 


3010 


4000 

the  task  require  the 


